Conventional soft partition based clustering algorithms usually cannot achieve desired clustering outcomes in the situations where the data are quite spare or distorted. To address this problem, based on maximum entropy clustering, by means of the strategy of historical knowledge learning, two novel soft partition based clustering models called SPBC-RHK-1 and SPBC-RHK-2 for short respectively were proposed. SPBC-RHK-1 is the basic model which only refers to the historical cluster centroids, whereas SPBC-RHK-2 is of advanced modality based on the combination of historical cluster centroids and historical memberships. In terms of the historical knowledge, the effectiveness of both algorithms was improved distinctly, and SPBC-RHK-2 method showed better effectiveness and robustness compared to the other method since its higher ability of utilizing knowledge. In addition, because the involved historical knowledge does not expose the historical raw data, both of these two approaches have good capacities of privacy protection for historical data. Finally, experiments were conducted on both artificial and real-world datasets to verify above merits.